Non-line-of-sight radar-based gesture recognition

ABSTRACT

This document describes techniques and devices for non-line-of-sight radar-based gesture recognition. Through use of the techniques and devices described herein, users may control their devices through in-the-air gestures, even when those gestures are not within line-of-sight of their device&#39;s sensors. Thus, the techniques enable users to control their devices in many situations in which control is desired but conventional techniques do permit effective control, such as to turn the temperature down in a room when the user is obscured from a thermostat&#39;s gesture sensor, turn up the volume on a media player when the user is in a different room than the media player, or pause a television program when the user&#39;s gesture is obscured by a chair, couch, or other obstruction.

PRIORITY APPLICATION

This application is a continuation of Ser. No. 14/582,896 titled“Non-Line-of-Sight Radar-Based Gesture Recognition,” and filed on Dec.24, 2014 and claims priority under 35 U.S.C. § 119(e) to U.S. PatentApplication Ser. No. 62/059,099, titled “Non-Line-of-Sight Radar-BasedGesture Recognition,” and filed on Oct. 2, 2014, the disclosure of whichis incorporated by reference herein in its entirety.

BACKGROUND

As smart devices proliferate in homes, automobiles, and offices, theneed to seamlessly and intuitively control these devices becomesincreasingly important. For example, users desire to quickly and easilycontrol their media players, televisions, and climate devices fromwherever they happen to be. Current techniques for controlling smartdevices, however, fail to provide seamless and intuitive control,instead relying on touch screens, hand-held remote controls, and clumsyaudio interfaces.

This background description is provided for the purpose of generallypresenting the context of the disclosure. Unless otherwise indicatedherein, material described in this section is neither expressly norimpliedly admitted to be prior art to the present disclosure or theappended claims.

SUMMARY

This document describes techniques and devices for non-line-of-sightradar-based gesture recognition. Through use of the techniques anddevices described herein, users may control their devices throughin-the-air gestures, even when those gestures are not withinline-of-sight of their device's sensors. Thus, the techniques enableusers to control their devices in many situations in which control isdesired but conventional techniques do permit effective control, such asto turn the temperature down in a room when the user is obscured from athermostat's gesture sensor, turn up the volume on a media player whenthe user is in a different room than the media player, or pause atelevision program when the user's gesture is obscured by a chair,couch, or other obstruction.

This summary is provided to introduce simplified concepts relating tonon-line-of-sight radar-based gesture recognition, which is furtherdescribed below in the Detailed Description. This summary is notintended to identify essential features of the claimed subject matter,nor is it intended for use in determining the scope of the claimedsubject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of techniques and devices for non-line-of-sight radar-basedgesture recognition are described with reference to the followingdrawings. The same numbers are used throughout the drawings to referencelike features and components:

FIG. 1 illustrates an example environment in which non-line-of-sightradar-based gesture recognition can be implemented, including though useof direct and reflected radar fields.

FIG. 2 illustrates the smart device of FIG. 1 in greater detail.

FIG. 3 illustrates an example penetration radar field.

FIG. 4 illustrates an example method enabling non-line-of-sightradar-based gesture recognition using a reflected radar field.

FIG. 5 illustrates a room of FIG. 1, including volumes that do not haveline-of-sight to a radar system and transmission paths through which areflected radar field is provided within those volumes.

FIG. 6 illustrates an example method enabling non-line-of-sightradar-based gesture recognition effective to control or communicate witha smart device that does not have line-of-sight to a user.

FIG. 7 illustrates a ground floor of a home having multiple radarsystems, some of which do not have line-of-sight to a user in the home.

FIG. 8 illustrates an example device embodying, or in which techniquesmay be implemented that enable use of, non-line-of-sight radar-basedgesture recognition.

DETAILED DESCRIPTION

Overview

This document describes techniques and devices enablingnon-line-of-sight radar-based gesture recognition. These techniques anddevices enable greater control of smart devices through recognizinggestures when those gestures are made without being within line-of-sightof a smart device's sensors.

Consider, for example, a case where a user has many smart devices in hishome. Assume that to control these devices he has numerous handheldremote controls; one for each device. Controlling these various smartdevices is impractical due to the number of remote controls needed.Further, even if the user had one remote control capable of controllingmultiple smart devices, he would still not be able to control all ofmultiple devices whenever he was not within line-of-sight of all ofthese various devices. And, even in the uncommon event of being inline-of-sight to many devices at once, the user still needs to carryaround a remote control.

Consider other conventional ways in which to control smart devices, suchas audio interfaces and line-of-sight cameras. Audio interfaces oftenfail to understand a user's intent and require the user to interrupt hisor her ongoing conversations. Line-of-sight cameras, while allowing auser to forgo carrying a remote control in some cases, requireline-of-sight and also can fail to provide robust and consistent gesturerecognition. These are but three of many example controllers that failto provide seamless and intuitive control.

In contrast, consider a case where a user is standing in her kitchen anddesires to pause her television that is in her living room, turn off hermicrowave because her dinner is getting too hot, and turn up the heat inher home. Assume that the user's hands are blocked from her microwave bysome hanging pots and pans and that her thermostat and television arewithin other rooms. The techniques described herein enable her tocontrol all three of these devices, even though two are in other roomsand the third is obscured by various objects. None of these haveline-of-sight to her, yet she can make a gesture to control each andevery one of these three different devices seamlessly and intuitively.

This is but one way in which non-line-of-sight radar-based gesturerecognition can be performed. This document now turns to an exampleenvironment, after which non-line-of-sight gesture recognition systems,example methods, and an example computing system are described.

Example Environment

FIG. 1 is an illustration of an example environment 100 in whichtechniques enabling non-line-of-sight radar-based gesture recognitioncan be performed. Environment 100 includes a room 102, a smart device104, and a user 106. Room 102 includes obstructions 108, walls 110, afloor 112 (shown with squares), and a ceiling 114 (transparent intop-down view, show with circles), which are described in more detailbelow. Room 102 is shown in two identical illustrations (102-1 and102-2) with the exception of one having a direct radar field and anotherhaving a reflected radar field, as noted below.

Smart device 104 includes non-line-of-sight gesture recognition system116 (NLOS system 116), which provides a direct radar field 118. Directradar field 118 does not have line-of-sight to some portions of room102, here volumes 120. A first volume 120-1 does not have line-of-sightto NLOS system 116 due to couch 108-1, which is one of obstructions 108.A second volume 120-2 does not have line-of-sight to NLOS system 116 dueto planter box 108-2, which is another of obstructions 108.

NLOS recognition system 116 is also capable of providing a reflectedradar field 122 effective to enable recognition of gestures within someor all of volumes 120 in which direct line-of-sight is not available.This reflected radar field 122 is shown provided through radartransmission lines 124, though additional description of transmissionlines is provided elsewhere herein. For visual clarity radar fields 118and 122 are shown separately, though both radar fields can be usedtogether.

While not shown in FIG. 1, NLOS system 116 can also provide apenetration radar field. This field is configured to penetrate variousmaterials, such as wool, cotton, nylon, or leather, but reflect fromhuman tissue, thereby further enabling recognition of gestures that arepartially or fully not in line-of-sight or otherwise obscured from NLOSsystem 116.

In more detail, consider FIG. 2, which illustrates smart device 104.Smart device 104 includes one or more computer processors 202 andcomputer-readable media 204 (e.g., memory media and storage media).Applications and/or an operating system (not shown) embodied ascomputer-readable instructions on computer-readable media 204 can beexecuted by processors 202 to provide some of the functionalitiesdescribed herein. Computer-readable media 204 also includes fieldmanager 206 (described below).

Smart device 104 may also include network interfaces 208 forcommunicating data over wired, wireless, or optical networks. By way ofexample and not limitation, network interface 208 may communicate dataover a local-area-network (LAN), a wireless local-area-network (WLAN), apersonal-area-network (PAN), a wide-area-network (WAN), an intranet, theInternet, a peer-to-peer network, point-to-point network, a meshnetwork, and the like. Smart device 104 may also include a display 210,though this is not required.

Smart device 104 also includes or has access to NLOS system 116, asnoted above, which is configured to provide radar fields by which tosense gestures. To enable this, NLOS system 116 includes a microwaveradio element 212, an antenna element 214, and a signal processor 216.Generally, microwave radio element 212 is configured to provide a radarfield having a direct radar field and a reflected radar field as notedabove, though a penetration radar field may also be included. Whileexamples shown herein generally show one NLOS system 116 per device,multiples can be used, thereby increasing coverage of a volume (e.g.,room 102), as well as a number, complexity, accuracy, resolution, androbust recognition of gestures.

Microwave radio element 212 can be configured to emit one or multiplesets of continuously modulated radiation, ultra-wideband radiation, orsub-millimeter-frequency radiation. Microwave radio element 212, in somecases, is configured to form radiation in beams, the beams aidingantenna element 214 and signal processor 216 to determine which of thebeams are interrupted, and thus locations of interactions (e.g., by ahand) within the radar field. In more detail, microwave radio element212 can be configured to emit microwave radiation in a 1 GHz to 300 GHzrange, as well as 57 GHz to 63 GHz, to provide the radar field. Thisrange affects antenna element 214's ability to sense interactions, suchas to track locations of two or more targets to a resolution of abouttwo to about 25 millimeters. Microwave radio element 212 can beconfigured, along with other entities of NLOS system 116, to have arelatively fast update rate, which can aid in resolution of theinteractions. By selecting particular frequencies, NLOS system 116 canoperate to provide a direct radar field to reflect from line-of-sightgestures, a reflected radar field to reflect first from some object orobjects and then from non-line-of-sight gestures, and/or a penetrationradar field to penetrate clothing and similar materials while reflectingfrom human tissue.

Antenna element 214 is configured to receive reflections of, or senseinteractions in, the radar field, and signal processor 216 is configuredto process the reflections or sensed interactions in the radar fieldsufficient to provide gesture data usable to determine a gesture fromthe sensed interactions. Antenna element 214 can include one or manysensors, such as an array of radiation sensors, the number in the arraybased on a desired resolution and whether the radar field is direct,reflected, or penetration. Alternately or additionally, antenna element214 may include one or many antennas, such as an array of antennas,directional or otherwise, coupled with respective radiation sensorsand/or signal processor 216.

Further, antenna element 214 or signal processor 216 can be configuredto differentiate between interactions in the radar field that are fromradar transmissions that are direct, through a material, or after beingreflected. Thus, a media player having NLOS system 116 may differentiatebetween radar reflecting directly from a user's hand to radar that wasfirst bounced off of a ceiling (e.g., a painted, gypsum drywall ceiling)and then reflected off of the user's hand.

The radar field provided by microwave radio element 212 can include oneor multiple three-dimensional (3D) volumes, planes, or surfaces (e.g., asurface applied to a user's hand) In each of these cases, antennaelement 214 is configured to sense interactions of one or multipletargets (e.g., two hands, fingers of one or two hands, etc.), and signalprocessor 216 is configured to process the sensed interactionssufficient to provide gesture data usable to determine gestures in threedimensions.

With the provided radar field, such as direct radar field 118 andreflected radar field 124 of FIG. 1, a user may perform complex orsimple gestures with a hand or device (e.g., a stylus) that interruptsthe field. Example gestures include the many gestures usable withcurrent touch-sensitive displays, such as swipes, two-finger pinch andspread, tap, and so forth. Other gestures are enabled that are complex,or simple but three-dimensional, examples include the many sign-languagegestures, e.g., those of American Sign Language (ASL) and other signlanguages worldwide. A few of these include an up-and-down fist, whichin ASL means “Yes”, an open index and middle finger moving to connect toan open thumb, which means “No”, a flat hand moving up a step, whichmeans “Advance”, a flat and angled hand moving up and down, which means“Afternoon”, clenched fingers and open thumb moving to open fingers andan open thumb, which means “taxicab”, an index finger moving up in aroughly vertical direction, which means “up”, and so forth. These arebut a few of many gestures that can be mapped to particular devices orapplications, such as the “Advance” gesture to skip to another songbeing played by a media player through a non-line-of-sight gesture.

As noted above, NLOS system 116 can provide a penetration radar fieldcapable of penetrating some objects and materials and then reflect offof human tissue. This is illustrated at FIG. 3, which shows hand 302having a penetration radar field 304 contacting a surface 306 of hand302. This penetration radar field 304 penetrates chair 308 to then bereflected from surface 306 and received back at antenna element 214 ofNLOS system 116. Microwave radio element 212 (not shown) providespenetration radar field 304, while antenna element 214 is configured toreceive a reflection caused by an interaction on surface 306 of hand 302that penetrates and then reflects back through chair 308, at which pointsignal processor 216 is configured to process the received reflectionsufficient to provide gesture data usable to determine a gesture.

Gestures can be associated with various smart devices, such as a deviceother than smart device 104. To do so, gestures can map to apre-configured control gesture associated with a control input for anapplication associated with any of these devices. Thus, a gesture can beset to control the application and/or device based on the gestureitself, rather than first selecting which device or application to whichthe control is intended. For example, many complex and unique gesturescan be recognized by NLOS system 116, thereby permitting precise and/orsingle-gesture control, even for multiple applications. Mapping thesemany gestures to various different devices and application permitscontrol, with or without explicitly selecting (usually through agesture) which device is intended to be controlled.

Smart device 104 is illustrated with various non-limiting exampledevices: smartphone 104-1, refrigerator 104-2, microwave oven 104-3,laptop 104-4, tablet 104-5, and television 104-6, though other devicesmay also be used, such as climate control devices (e.g., a thermostat orceiling fan), wearable devices, desktop computers, netbooks, e-readers,cameras, automotive systems, and so forth.

NLOS system 116 also includes a transceiver 218 configured to transmitgesture data to a remote device, such as in cases where NLOS system 116is not integrated with smart device 104. Gesture data can be provided ina format usable by the receiving device sufficient to recognize agesture using the gesture data.

NLOS system 116 may also include one or more system processors 220 andsystem media 222 (e.g., one or more computer-readable storage media).System media 222 includes system manager 224, which can perform variousoperations, including determining a gesture based on gesture data fromsignal processor 216, mapping the determined gesture to a pre-configuredcontrol gesture associated with a control input for an applicationassociated with a remote smart device, and causing transceiver 218 totransmit the control input to the remote smart device effective toenable control of the application or device. This is but one of the waysin which the above-mentioned control through NLOS system 116 can beenabled. Note also that while NLOS system 116 is shown integral withsmart device 104, it can instead be part of a device having few or nocomputing capabilities and still provide gesture data to be recognizedand/or mapped to a control input. Operations of NLOS system 116, systemmanager 224, and field manager 206 are provided in greater detail aspart of methods 400 and 600 below.

These and other capabilities and configurations, as well as ways inwhich entities of FIGS. 1-3 act and interact, are set forth in greaterdetail below. These entities may be further divided, combined, and soon. The environment 100 of FIG. 1 and the detailed illustrations ofFIGS. 2 and 3 illustrate some of many possible environments and devicescapable of employing the described techniques.

Example Methods

FIGS. 4 and 6 depict methods 400 and 600. Method 400 enablesnon-line-of-sight radar-based gesture recognition using a reflectedradar field, and can be performed separate from or integrated in wholeor in part with method 600. Method 600 enables non-line-of-sightradar-based gesture recognition where a direct line-of-sight is notpresent for one radar system but a gesture is recognized through anotherradar system and then passed to the other radar system through direct orindirect radar fields.

These methods are shown as sets of operations (or acts) performed butare not necessarily limited to the order or combinations in which theoperations are shown herein. Further, any of one or more of theoperations may be repeated, combined, reorganized, or linked to providea wide array of additional and/or alternate methods. In portions of thefollowing discussion reference may be made to environment 100 of FIG. 1and entities and examples detailed in FIGS. 2, 3, 5, and 7, reference towhich is made for example only. The techniques are not limited toperformance by one entity or multiple entities operating on one device.

At 402, a volume of a room in which there is no line-of-sight to a radarsystem is determined. The techniques may determine that there is such avolume in various manners, such as through tracking a moving entity, acamera, bounced (e.g., reflected) radar transmissions, and so forth.

For example, field manager 206 may track, with a radar transmission, aperson or animal's movement in the room. If, during a portion of theperson or animal's movement field manager 206 does not receive areflection from the person or animal, field manager 206 may determine,based on the person or animal's projected path or a part of the personor animal no longer trackable (e.g., a person's legs walking behind acouch), that a volume in the room exists where direct line-of-sight isnot available.

Field manager 206 may also or instead use a camera or other imagesensor, using radar or otherwise. Field manager 206 captures an imageusing the camera to provide an image of the room from a perspective ofNLOS system 116. Field manager 206 then receives or determines (e.g.,through sonar or radar reflection) dimensions for the room (e.g., 4meters wide, 7 meters deep, 3 meters high). With these dimensions andthe image of the room, non-imaged but existing floor, wall, or ceilingareas can be determined. With this data, field manager 206 may thendetermine the volume of the room in which there is no line-of-sight toNLOS system 116.

Further still, field manager 206 may determine non-line-of-sight volumesbased on obstructions. Thus, on providing a direct radar field toreceive reflections from objects in the room, field manager 206 assumesthat regions behind obstructions are likely to be volumes in which apotential gesture may be made, and in any case, these are very likely topart of the room and do not have line-of-sight. As part of thisdetermination obstructions are objects other than floors, walls, orceilings of the room. Some objects reflect radar and some are thosethrough which radar may pass. Because of this, portions of the objectspenetrated by the radar field can reduce the size of the volume.

At 404, a set of transmission paths sufficient to cause a reflectedradar field to be provided within at least a portion of the volume isdetermined by providing radar transmissions from the radar system intothe room. As noted in part above, determining a volume in which noline-of-sight is available may include radar reflection, in which casesome transmission paths may be known. In many cases, however, while thevolume is known the transmission paths to cause a reflected radar fieldare not yet known.

To determine these transmission path, field manager 206 may iterativelyprovide radar transmissions and receive reflections from the providedradar transmissions. Some of these radar transmissions may not passthrough the determined volume, and thus are not needed. Some others,however, pass through one of the determined volumes and are thencorrelated with that volume. In still further cases, a person or animalpasses through the determined volume and radar transmission areiteratively provided and then, on reflection from the person or animal,correlated to the volume. In so doing various types and frequencies ofradar transmissions can be used, including narrow-beam transmission andreflection.

By way of example, consider FIG. 5, which illustrates room 102 of FIG.1, including smart device 104, user 106, obstructions 108, walls 110,floor 112, ceiling 114, NLOS system 116, and volumes 120-1 and 120-2.Two transmission paths are shown, wall-and-object-reflection path 502and ceiling-and-floor path 504. These illustrate but two of manydifferent transmission paths capable of providing reflection radar field122 (shown in FIG. 1 and omitted for clarity in FIG. 5).

Wall-and-object-reflection path 502 includes eight portions, eachreflecting off of, or resulting from a reflection off of, a wall orobject. Thus, path 502 proceeds from NLOS system 116 to one of walls110, then again off of another of walls 110, then off of a back of couch108-1, back to the other of walls 110, to couch 108-1, to the other ofthe walls 110, than off planter box 108-2, and then back to NLOS system116. Some transmission paths do not result in a reflection returning,and thus can be forgone, while others pass only within volumes that arein line-of-site of NLOS system 116, and can also be forgone. Path 504proceeds from NLOS system 116 to ceiling 114 (shown reflected at acircle), then to one of walls 110, off floor 112 (shown reflected at asquare), off ceiling 114, and then back to NLOS system 116. These twopaths 502, 504 provide but two example transmission paths by which NLOSsystem 116 and field manager 206 may provide a reflected radar field(e.g., 122 of FIG. 1) in a volume in which line-of-sight to NLOS system116 is not available. As noted, this reflected radar field enablesrejection of gestures made within some or all of volumes 120.

At 406, a radar field is provided within the one or more volumes withinthe room, the radar field including at least the reflected radar field.In some cases, the radar field also includes a direct radar field or apenetration radar field as noted herein. By so doing, a radar fieldusable to recognize gestures can be provided for gestures that do nothave direct line-of-sight from a radar system.

At 408, a reflection of a gesture interaction made within the radarfield within the room is received. This reflection enables capture ofdata about the gesture interaction that can then be used to determinethe gesture being made. This reflection can be within a reflected,direct, or penetration radar field as noted above.

At 410, a gesture is determined based on captured data associated withthe reflection of the gesture interaction. The gesture determined can beone of many gestures noted above, including those mapping directly to adevice or application and a control command, such as to mute atelevision set.

At 412, the determined gesture is provided to a device or applicationeffective to control the device or application. As part of thisprovision, field manager 206 may determine to which device orapplication to send the control, such as by analyzing a map of gesturesto various devices and applications.

FIG. 6 illustrates method 600, which enables non-line-of-sightradar-based gesture recognition effective to control or communicate witha smart device that does not have line-of-sight to a user.

At 602, a radar field is provided, such as a field having one or more ofa direct, reflected, or penetration radar field, as described above.This radar field can be provided by a radar system that does or does nothave line-of-sight to a user making a gesture.

At 604, reflections from an interaction that is within the radar fieldare sensed through a radar system. This radar system is assumed to haveline-of-sight, or is able to compensate for not having line-of-sight,effective to receive reflections from a user. Another smart device,however, which may or may not have a radar system, does not haveline-of-sight to the user and is not able to compensate. Thus, bothdevices may not have line-of-sight to a user but one of those devices isable to sense reflections, such as through a penetration or reflectionradar field.

By way of example, consider FIG. 7, which illustrates a ground floor 702of a home. This home includes four rooms 704, media room 704-1, pianoroom 704-2, living room 704-3, and kitchen/dining room 704-4. As shown,there are five radar systems 706, a media-player 708's radar system706-1, a thermostat 710's radar system 706-2, a television 712's radarsystem 706-3, an oven 714's radar system 706-4, and a microwave-oven716's radar system 706-5. Multiple scenarios in which the techniquesenable gesture recognition without line-of-sight to a smart deviceintended to be controlled are shown in FIG. 7, such as when user 718wants to control television 712 while in kitchen 704-4. To do so, fieldmanager 206 receives reflections from user 718 at an NLOS system 116 ofone of radar systems 706 that is not in the same room as user 718.

By way of a particular example, assume that user 718 is inkitchen/dining room 704-4 and wants to turn down the heat in the houseby controlling thermostat 710, which is in piano room 704-2. To do so,user 718 makes a gesture to turn down the heat while standing inkitchen/dining room 704-4, which, at operation 604, an NLOS system 116operating in oven 714's radar system 706-4 receives. The gesture isreceived by receiving user 718's reflection within radar field 720 at anantenna of NLOS system 116.

At 606, a gesture is recognized based on the reflections received. Thiscan be performed at a smart device associated with a radar system atwhich the reflection is received, or gesture data for the reflection canbe received at a smart device intended to be controlled. Thus, fieldmanager 206 or system manager 224 may receive the reflection and passgesture data for the reflection to other smart devices, such as to allof the other radar systems and/or devices in a home or office. In such acase, the smart devices can determine if the control is intended forthem based on the gesture determined from the gesture data, such ashaving received a prior gesture selecting to control that device orreceiving a gesture associated with the smart device.

At 608, the gesture is determined to be associated with (or notassociated with) a non-line-of-sight smart device. Generally, fieldmanager 206 may determine that the gesture is associated with anon-line-of-sight smart device by mapping the gesture to apre-configured gesture intended to establish communication with thenon-line-of-sight smart device. In some cases this pre-configuredgesture establishes communication with the non-line-of-sight smartdevice effective to enable a future gesture determined at the smartdevice to control the non-line-of-sight smart device. In some othercases, field manager 206 may determine that the gesture is associatedwith the non-line-of-sight smart device by mapping the gesture to aunique gesture. This unique gesture can be of a set of unique gesturesuniquely mapped to each of a set of smart devices within some group,such as smart devices on ground floor 702 of FIG. 7.

Continuing the ongoing example of FIG. 7, assume that field manager 206operating at NLOS system 116 of oven 714's radar system 706-4 recognizesa gesture made by user 718. This gesture may then be determined toindicate control of thermostat 710, after which another gesture isrecognized and then determined to be associated with thermostat 710based on the prior gesture indicating control of thermostat 710 isintended. Assume, for example, that a unique, complex gesture with aclosed hand and exposed thumb that moves up and the down is associatedwith thermostat 710. A second gesture, here assumed to be a non-uniquemagnitude gesture of a cupped hand moving clockwise orcounter-clockwise, indicates to turn up or down something, respectively.Because this second gesture is received soon after the first gestureindicating control of thermostat 710 is intended, it is then recognizedby repeating steps of method 600 and then passed (at operation 610) tothermostat 710 effective to turn the heat up or down. This repetition isshown at a dashed arrow in FIG. 6.

Similarly, if the first gesture is associated with control of mediaplayer 708, the second gesture indicating to turn up or down, wouldinstead turn up or turn down the volume of media player 708, all withoutline-of-sight from the intended smart device.

At 610, the gesture is passed to the non-line-of-sight smart device.Thus, the gesture can be passed from one smart device to another smartdevice effective to enable the gesture to control the non-line-of-sightsmart device or establish communication with the non-line-of-sight smartdevice.

The preceding discussion describes methods relating to non-line-of-sightradar-based gesture recognition. Aspects of these methods may beimplemented in hardware (e.g., fixed logic circuitry), firmware,software, manual processing, or any combination thereof. Thesetechniques may be embodied on one or more of the entities shown in FIGS.1-3, 5, 7, and 8 (computing system 800 is described with reference toFIG. 8 below), which may be further divided, combined, and so on. Thus,these figures illustrate some of the many possible systems orapparatuses capable of employing the described techniques. The entitiesof these figures generally represent software, firmware, hardware, wholedevices or networks, or a combination thereof.

Example Computing System

FIG. 8 illustrates various components of example computing system 800that can be implemented as any type of client, server, and/or smartdevice as described with reference to the previous FIGS. 1-7 toimplement non-line-of-sight radar-based gesture recognition. Inembodiments, computing system 800 can be implemented as one or acombination of a wired and/or wireless wearable device, System-on-Chip(SoC), and/or as another type of device or portion thereof. Computingsystem 800 may also be associated with a user (e.g., a person) and/or anentity that operates the device such that a device describes logicaldevices that include users, software, firmware, and/or a combination ofdevices.

Computing system 800 includes communication devices 802 that enablewired and/or wireless communication of device data 804 (e.g., receiveddata, data that is being received, data scheduled for broadcast, datapackets of the data, etc.). Device data 804 or other device content caninclude configuration settings of the device, media content stored onthe device, and/or information associated with a user of the device.Media content stored on computing system 800 can include any type ofaudio, video, and/or image data. Computing system 800 includes one ormore data inputs 806 via which any type of data, media content, and/orinputs can be received, such as human utterances, interactions with aradar field, user-selectable inputs (explicit or implicit), messages,music, television media content, recorded video content, and any othertype of audio, video, and/or image data received from any content and/ordata source.

Computing system 800 also includes communication interfaces 808, whichcan be implemented as any one or more of a serial and/or parallelinterface, a wireless interface, any type of network interface, a modem,and as any other type of communication interface. Communicationinterfaces 808 provide a connection and/or communication links betweencomputing system 800 and a communication network by which otherelectronic, computing, and communication devices communicate data withcomputing system 800.

Computing system 800 includes one or more processors 810 (e.g., any ofmicroprocessors, controllers, and the like), which process variouscomputer-executable instructions to control the operation of computingsystem 800 and to enable techniques for, or in which can be embodied,non-line-of-sight radar-based gesture recognition. Alternatively or inaddition, computing system 800 can be implemented with any one orcombination of hardware, firmware, or fixed logic circuitry that isimplemented in connection with processing and control circuits which aregenerally identified at 812. Although not shown, computing system 800can include a system bus or data transfer system that couples thevarious components within the device. A system bus can include any oneor combination of different bus structures, such as a memory bus ormemory controller, a peripheral bus, a universal serial bus, and/or aprocessor or local bus that utilizes any of a variety of busarchitectures.

Computing system 800 also includes computer-readable media 814, such asone or more memory devices that enable persistent and/or non-transitorydata storage (i.e., in contrast to mere signal transmission), examplesof which include random access memory (RAM), non-volatile memory (e.g.,any one or more of a read-only memory (ROM), flash memory, EPROM,EEPROM, etc.), and a disk storage device. A disk storage device may beimplemented as any type of magnetic or optical storage device, such as ahard disk drive, a recordable and/or rewriteable compact disc (CD), anytype of a digital versatile disc (DVD), and the like. Computing system800 can also include a mass storage media device 816.

Computer-readable media 814 provides data storage mechanisms to storedevice data 804, as well as various device applications 818 and anyother types of information and/or data related to operational aspects ofcomputing system 800. For example, an operating system 820 can bemaintained as a computer application with computer-readable media 814and executed on processors 810. Device applications 818 may include adevice manager, such as any form of a control application, softwareapplication, signal-processing and control module, code that is nativeto a particular device, a hardware abstraction layer for a particulardevice, and so on.

Device applications 818 also include any system components, engines, ormanagers to implement non-line-of-sight radar-based gesture recognition.In this example, device applications 818 include field manager 206 andsystem manager 224.

CONCLUSION

Although embodiments of techniques using, and apparatuses enabling,non-line-of-sight radar-based gesture recognition have been described inlanguage specific to features and/or methods, it is to be understoodthat the subject of the appended claims is not necessarily limited tothe specific features or methods described. Rather, the specificfeatures and methods are disclosed as example implementations enablingnon-line-of-sight radar-based gesture recognition.

What is claimed is:
 1. A smart device associated with a radar field, thesmart device comprising: a non-line-of-sight gesture-recognition systemconfigured to: provide the radar field, the radar field including adirect radar field and a reflected radar field, the reflected radarfield provided in one or more volumes in a room to which directline-of-sight from the non-line-of-sight gesture-recognition system isnot available; sense radar reflections from an interaction that iswithin the reflected radar field, the radar reflections sensed byreceiving the radar reflections from a surface of the room and at anantenna of the non-line-of-sight gesture-recognition system; recognize,based on the radar reflections from the interaction that is within thereflected radar field, a gesture; and determine that the gesture isassociated with a non-line-of-sight smart device; and a transceiver ornetwork interface configured to pass the gesture from the smart deviceassociated with the radar field to the non-line-of-sight smart device ina format usable by the non-line-of-site smart device to recognize thegesture and effective to enable the gesture to control thenon-line-of-sight smart device or establish communication with thenon-line-of-sight smart device.
 2. The smart device of claim 1, wherein:the radar field further includes a penetration radar field; thepenetration radar field is provided in one or more volumes in a room towhich direct line-of-sight from the non-line-of-sightgesture-recognition system is not available; and the non-line-of-sightgesture-recognition system is further configured to sense additionalradar reflections from a second interaction that is within thepenetration radar field.
 3. The smart device of claim 2, wherein thenon-line-of-sight gesture-recognition system is further configured tosense the additional radar reflections from the second interaction thatis within the penetration radar field by receiving the additional radarreflections at the antenna of the non-line-of-sight gesture-recognitionsystem and through an object occluding the second interaction that iswithin the penetration radar field.
 4. The smart device of claim 1,wherein: the non-line-of-sight recognition system is further configuredto receive a second gesture from a second non-line-of-sightgesture-recognition system, the second non-line-of-sightgesture-recognition system is associated with the non-line-of-sightsmart device, the second gesture is associated with the smart device,and the non-line-of-sight gesture-recognition system is furtherconfigured to control the smart device based on the second gesture. 5.The smart device of claim 1, wherein the non-line-of-sight recognitionsystem is further configured to determine that the gesture is associatedwith the non-line-of-sight smart device by mapping the gesture to apre-configured control gesture associated with a control input for anapplication associated with the non-line-of-sight smart device.
 6. Thesmart device of claim 1, wherein the non-line-of-sight recognitionsystem is further configured to determine that the gesture is associatedwith the non-line-of-sight smart device by mapping the gesture to apre-configured communication-establishment gesture associated with thenon-line-of-sight smart device.
 7. The smart device of claim 1, whereinthe non-line-of-sight recognition system is further configured todetermine that the gesture is associated with the non-line-of-sightsmart device by mapping the gesture to a unique gesture of a set ofunique gestures unique to each of a set of smart devices.
 8. The smartdevice of claim 1, wherein the transceiver or network interface isfurther configured to pass the gesture from the smart device to thenon-line-of-sight smart device by passing the gesture as gesture datathat maps the gesture to a pre-configured control gesture for anapplication executing at the non-line-of-sight smart device.
 9. Thesmart device of claim 1, wherein the format usable by thenon-line-of-site smart device to recognize the gesture is furthereffective to enable the non-line-of-sight smart device or an applicationexecuting at the non-line-of-sight smart device to determine if thegesture maps to a pre-configured control gesture for an applicationexecuting at the non-line-of-sight smart device.
 10. A smart deviceassociated with a radar field, the smart device comprising: anon-line-of-sight gesture-recognition system configured to: provide theradar field; sense radar reflections from an interaction that is withinthe radar field; recognize, based on the radar reflections from theinteraction that is within the radar field, a gesture; and determinethat the gesture is associated with a non-line-of-sight smart device;and a transceiver or network interface configured to: pass the gesturefrom the smart device associated with the radar field to thenon-line-of-sight smart device in a format usable by thenon-line-of-site smart device to recognize the gesture and effective toestablish communication with the non-line-of-sight smart device; andestablish communication with the non-line-of-sight smart deviceresponsive to a determination that the gesture is for establishingcommunication with the non-line-of-sight smart device to enable controlof the non-line-of-sight smart device through a future gesturedetermined at the smart device and sensed through the non-line-of-sightgesture-recognition system of the smart device.
 11. The smart device ofclaim 10, wherein the non-line-of-sight recognition system is furtherconfigured to determine that the gesture is associated with thenon-line-of-sight smart device by mapping the gesture to apre-configured control gesture associated with a control input for anapplication associated with the non-line-of-sight smart device.
 12. Thesmart device of claim 10, wherein the non-line-of-sight recognitionsystem is further configured to determine that the gesture is associatedwith the non-line-of-sight smart device by mapping the gesture to apre-configured communication-establishment gesture associated with thenon-line-of-sight smart device.
 13. The smart device of claim 10,wherein the non-line-of-sight recognition system is further configuredto determine that the gesture is associated with the non-line-of-sightsmart device by mapping the gesture to a unique gesture of a set ofunique gestures unique to each of a set of smart devices.
 14. The smartdevice of claim 10, wherein the transceiver or network interface isfurther configured to pass the gesture from the smart device to thenon-line-of-sight smart device by passing the gesture as gesture datathat maps the gesture to a pre-configured control gesture for anapplication executing at the non-line-of-sight smart device.
 15. Thesmart device of claim 10, wherein the format usable by thenon-line-of-site smart device to recognize the gesture is furthereffective to enable the non-line-of-sight smart device or an applicationexecuting at the non-line-of-sight smart device to determine if thegesture maps to a pre-configured control gesture for an applicationexecuting at the non-line-of-sight smart device.
 16. A systemcomprising: a non-line-of-sight gesture-recognition system configuredto: provide a radar field within a room, the radar field includingreflected radar fields, the reflected radar fields being reflected offof one or more surfaces of the room and provided in one or more volumesin the room to which direct line-of-sight from the non-line-of-sightgesture-recognition system is not available; and receive a radarreflection of a gesture interaction made within the one or more volumesin the room to which direct line-of-sight from the non-line-of-sightgesture-recognition system is not available, the radar reflectionenabling capture of data about the gesture interaction, one or moreprocessors; and one or more computer readable storage media havinginstructions stored thereon that, responsive to execution by the one ormore processors, implement a system manager, the system managerconfigured to: determine a gesture based on the data about the gestureinteraction; and provide the gesture to a device or applicationeffective to control the device or application.
 17. The system of claim16, wherein the radar reflection of the gesture interaction is madewithin one of the reflected radar fields and the radar reflection isreceived by an antenna of the non-line-of-sight gesture-recognitionsystem from a second radar reflection off of one of the one or moresurfaces of the room.
 18. The system of claim 16, wherein thenon-line-of-sight gesture-recognition system is further configured toprovide the radar field within the room further by providing penetrationradar fields within portions of the one or more volumes of the room towhich direct line-of-sight from the non-line-of-sightgesture-recognition system is not available.
 19. The system of claim 18,wherein the radar reflection of the gesture interaction is made withinone of the penetration radar fields.
 20. The system of claim 16, whereinthe non-line-of-sight recognition system is further configured todetermine that the gesture is associated with the non-line-of-sightsmart device by mapping the gesture to: a pre-configured control gestureassociated with a control input for an application associated with thenon-line-of-sight smart device; a pre-configuredcommunication-establishment gesture associated with thenon-line-of-sight smart device; or a unique gesture of a set of uniquegestures unique to each of a set of smart devices.